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SENSORY COMMUNICATION APPARATUS 

5 

This invention is concerned with a sensory communication 
apparatus, and in particular to such apparatus for the dynamic 
display to a blind person of textual and graphical information. 

10 Maps are used for three main purposes, namely for education, 

such as the study of geography, for planning a journey, and for 
navigation during a journey. 

The tasks of exploration are similar in the three cases and 
15 are essentially to discover the locations, names and 
characteristics of, and relationships between, real-world objects 
or features, both natural and man-made. The goal of exploration 
may be to establish the layout of a place, or more specifically 
to find a suitable route from one place to another. 

20 

There have been a number of developments in making maps 
accessible for blind people. Firstly there are the maps which 
are purely tactile and in which writing is in Braille, as raised 
dots, and texture is used as a substitute for colour. Such maps 

25 are "crude" in that there can be little detail, and resolution 
is small. Thus only a limited amount of information can be 
presented for a given size of map - several orders of magnitude 
less than a conventional map used by sighted people. There are 
problems of producing tactile maps, though there have been 

30 technical developments to improve the situation. The cost of 
reproduction (i.e. the per copy cost) is still high. 

Similar problems arise in the presentation of other multi- 
dimensional information including graphs, charts, block diagrams, 
35 tables and matrices. Even with a simple table the single line 
display of a conventional dynamic Braille apparatus can lead to 
confusion becayse the line crosses the columns. 
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There have been various attempts to get away from a 
5 "hard-copy" approach and substitute a dynamic tactile display, 
using a physical surface which can be altered under control of 
a computer. These displays are expensive because the surface is 
composed of a large array of movable elements, but the cost of 
map reproduction (i.e. producing copies of a map to distribute) 
10 is negligible since the maps are in electronic form. 

It is an object of the present invention to obviate or 
mitigate these difficulties. 

15 The present invention is sensory communication apparatus 

comprising a central processing unit having a data store, an 
output processor, means for controlling the output of data from 
the data store to the output processor, and output means 
connected to receive data from the processor, the output means 

20 being responsive to the output data whereby the output of the 
computer can be determined by one or more of the user's senses 
of touch, sound or sight. 

The means for controlling the output of the data store may 
25 comprise a pointing device. 

Preferably the output means includes tactile elements in the 
form of pins, each having associated with it an electro- 
mechanical transducer. 

30 

Preferably the pins are also associated with respective 
switches which are connected through an input processor to the 
central processing unit. 

35 The pins may be arranged in pairs each for simultaneous 

contact by a respective finger and each pin of each pair can be 
vibrated at different frequencies or with different pulse 
lengths. 
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The output means may be operable in either a character mode 
5 or a surface mode. 

Preferably the patterns in which the pins are actuated are 
related to text characters. 

10 The apparatus may include a speech synthesiser responsive 

to data output from the central processing unit. 

Preferably a keyboard or keypad is connected through an 
input processor to the central processing unit. 

15 

The keyboard or keypad may be operable in a character mode 
or a control mode. 

Graphical information may be held in the data store of the 
20 central processing unit as a virtual surface, over which moves 
a notional cursor. The data output reflects the graphical 
information under the cursor, and the relative position of 
graphical objects on the surface. A selected graphical object 
can be described to the user in text, which is output using the 
25 same tactile device, or using audio (speech) or visual display. 

Embodiments of the present invention will now be described, 
by way of example, with reference to the accompanying drawings, 
in which:- 

30 

Fig.l is a block circuit diagram of an embodiment of the 

present invention; 
Fig. 2 is a more detailed circuit diagram of part of 

Fig.l; 

35 Fig. 3 is a more detailed circuit diagram of part of 

Fig. 2; 

Fig. 4 is a side elevation of an input/output sensor; 
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Figs.5 is a plan view of a tactile input/output unit; and 
5 Fig. 6 is a side view of the tactile input/output unit of 

Fig. 5. 

Referring now to Fig.l, tactile communication apparatus 
according to an embodiment of the present invention comprises a 
central processing unit 10 having a data store 11 and being 
connected through an output processor 12 and an input processor 
14 to a number of tactile sensors 16 mounted on an input/output 
unit 18 (Fig. 5). The input and output processors need not, of 
course, be be separate units but may be provided in a single 
integrated circuit. 

The sensors 16 of this embodiment are shown in Fig. 4 and it 
should be noted that each sensor 16 is an input/ output device 
that both generates a mechanical movement in response to output 
20 signals received from the output processor 12 and generates 
signals in response to a mechanical input, the signals being 
passed to the central processing unit 10 through the input 
processor 14. 

Referring now to Fig. 4, each sensor 16 comprises a pair of 
pins 20, a pair of electro-mechanical transducers, in this 
embodiment piezo strips 22, and a pair of membrane, switches 24 
mounted on a printed circuit board 26 supported from the top 
plate 28 of the unit 18. The strips 22 are clamped at one end and 
mount the pins 20 at their other end, the pins 20 projecting 
upwardly through the top plate 28. The switches 24 are each 
mounted beneath a respective pin 20 to be closed by the piezo 
strip when the pin is depressed. 

In Fig. 2 the output processor 12 is shown to comprise, 
connected in cascade, a level translator 34, a pulse width 
modulator generator 36, a high voltage digital amplifier 38 and 
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a number of low pass filters 40 each connected to a respective 
5 piezo strip 22 in a sensor 16. In this embodiment twelve low pass 
filters 40 are provided. The output processor is powered from a 
low voltage source, preferably a battery, connected to a step-up 
voltage convertor 42 the high voltage output of which powers the 
processor more efficiently than would a low voltage. 

10 

The details of the pulse width modulation generator 36 are 
shown in Fig. 3. The generator 36 consists of four sources that 
may be mapped onto any combination of sixteen outputs. The 
sources are controlled by a control unit 50 which interprets the 
15 commands and data incoming from the computer 10 via the level 
translater 34 and controls the various aspects of pulse 
generation. 

Each of the four sources generates an arbitrary wave of a 
20 number of different amplitudes and frequencies and comprises a 
phase accumulator 52a, 52b , 52c, or 52d, which is set to step 
through a respective look-up table 54a, 54b 54c, or 54d, at a 
configurable rate whereby the frequency can be varied by altering 
the step and the waveform can be altered by using a number of 
25 different look-ups. The output from the look-up table is then 
scaled by a respective scaler 56a, 56b, 56c or 56d, according to 
the desired amplitude. 

The four current samples to be output are passed to their 
30 respective pulse width generators 58a-58d which create pulses 
which have widths proportional to the desired value. 

Any of the generators can be fed to any of the outputs. This 
is achieved by preloading a mask register 60a, 60b, 60c or 60d 
35 with the output pattern for the respective pulse width generator. 
When the generator is generating a pulse, this output word is fed 
through. When the generator is not generating a pulse, the output 
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mask is gated off and therefore does not play a part in the 
5 output . 

The sixteen outputs from each of the four generators are 
mixed together using an exclusive- OR gate 66. This results in 
a true mixing of the four sources when the resultant waveform has 
10 passed through its filtering process. 

In embodiments allowing access to maps or other graphical 
information, the array of transducers are mounted on, or are 
otherwise associated with, a pointing device, such as a computer 
15 mouse or a touch sensitive tablet. As the mouse moves, or a 
depression moves across the tablet, the window is correspondingly 
moved over the virtual surface representing the map or graphical 
image . 

20 To allow a map to be scaleable over a wide range of scales, 

the information can be stored at various levels of detail, with 
the greater detail suppressed for smaller-scale presentation. 
Thus for example, on a large scale the map may show the exact 
shapes of buildings, on a smaller scale building may be 

25 represented as simple rectangles, and on a smaller scale still 
the buildings may be merged into a single object representing a 
built-up area. In the current invention, the user can zoom in 
and out at will, which compensates for the small size of window. 

30 In Fig. 5 is shown the plan view of a tactile input/output 

unit 18 which is similar to a conputer mouse in that it has a 
mouse ball 88 (Fig. 6) on its underside and can therefore serve 
as a pointing device. In addition the unit 18 is provided with 
four tactile sensors (70, 72, 74, and 76) each having a pair of 

35 pins 20 projecting through its upper surface and two sensors 
having two pairs of pins 84, 86, as seen in Fig. 6, projecting 
through each of its sides 78 and 80. The sixteen pins 20 have 
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associated respective transducers in the form of piezo strips 22 
5 each connected to a respective one of the low pass filters 40 of 
Fig. 2. 

The unit is, in use, held in the hand with the four fingers 
engaging the four pairs of pins at its upper surface and the 
10 thumb engaging one of the two pairs of pins at the sides 
depending upon which hand is holding the unit. 

In embodiments of the invention there are two modes of 
operation, namely character and surface. 

15 

In character mode, output is by characters, each a pattern 
on the pins formed as follows. Each pair of pins has three states 
determined by the frequency of vibration of the piezo strips; in 
the first state one of the strips vibrates at a low frequency (20 

20 Hz), in the second state both piezo strips vibrate at the low 
frequency and in the third state both piezos vibrate at a higher 
frequency (200 Hz). With one of the four finger pairs on there 
are three times four states, i.e. twelve states. With one of the 
two thumb pairs on there are six states. The alphabet is coded 

25 by patterns comprising either a single finger state, or a single 
finger state combined, simultaneously or sequentially, with a 
single thumb state. 

In surface mode, the output corresponds to the virtual 
30 surface under the fingers. The piezo strips vibrate according to 
an algorithm based on frequency and distance from vectors forming 
virtual objects on the surface. 

In a modified embodiment the states of the pins are 
35 distinguished by the length of the vibration pulses as well as 
or instead of by the frequency of vibration. 
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Furthermore, embodiments of the invention have two input 
5 modes, namely character and control. 

In character mode the input is via keys, i.e the pins 20, 
acting on the input switches 24 mounted under the piezo strips. 
Patterns of input can be produced to correspond to patterns of 
10 output. Three input stimuli are possible per finger, 
corresponding to the three output states per finger, one 
pin/piezo depressed, the other depressed, and both depressed. 

In control mode, input is again via the keys and 
15 switches 24. A single keystroke is used for simple commands such 
as Next, Previous, Up, Down, Enter and Leave. These commands are 
used for navigating in information space, typically for exploring 
a document hierarchy, and for editing. The same commands are used 
at all levels in the structure of information space which is 
20 basically organised as a tree with hyperlinks. 

The patterns of activation of the pins can be used to give 
a direction, e.g. a compass bearing or the direction of an object 
from the cursor position. The patterns can also be used to 
25 indicate what is under the cursor, or in the immediate vicinity. 
By selecting objects, feeling the patterns and moving the 
pointer, the user is able to explore a map, or other graphical 
image, represented on the virtual surface. 

30 One form of the input/output unit has six pins with the 

associated transducers arranged as a hexagon about a seventh 
central pin.' The pins are used to guide the hand of the user 
holding the. tactile input/output unit in the direction 
corresponding to the direction of the graphical object from the 

35 cursor on the virtual surface. This allows the user to explore 
the surface for objects which the user has selected. There is 
means of selecting a single object or a group of objects with 
shared characteristics. 
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With this particular input/output unit, the method of 
5 finding a particular object and its shape is a follows. 

Consider first a point object such as bus stop. While the window 
is not over the object, the pin or pair of pins closest to the 
object are activated periodically, with a period proportional to 
distance. The user can then move the window towards the object, 

10 and the frequency of activation increases as the object is 
approached. When the window is directly over the point, the 
central pin is activated. Next consider a line object such as 
the centre line of a pavement along a street. While the window 
is not over the object, again the pin or pair of pins closest to 

15 the object are activated periodically. When the line is reached 
the pins over the line are activated. The user can then follow 
the line. While exactly over the line, the central pin is 
operated. Now consider objects which have an area (i.e. are not 
points lines). The same procedure is followed to find the edge 

20 of the object. However if the window is moved inside the area, 
the central pin is continuously activated, and the pins nearest 
to the nearest edge are periodically activated. 

A particularly useful aspect of the invention is the ability 
25 of the user to feel the input as it is being drawn. Thus a person 
can input a line onto a map by moving the pointing device and 
then immediately feel the line using the tactile means. 

For examining the relationship and shapes of different areas 
30 of the map, different areas are allocated different "colours" on 
the virtual surface, and each pin reacts to a different colour, 
as the cursor is moved over the surface. The user can thus scan 
the surface and detect the position of the different coloured 
areas. This is particularly efficient as no map requires more 
35 than four colours to distinguish different areas. 

Sounds may be produced corresponding to, pin activation 
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, above. Embodiments of the invention may have an audio output 
5 where the sounds corresponding to pin activation are combined 
with speech output of text information. 

While the main outputs are tactile or audio a visual output 
may also be provided as, for example, a word-by-word visual 

10 display of text with large characters, where the display or 
highlighting of each word is synchronised or partially 
synchronised with the speech output of the word such that the 
word is spoken immediately before, immediately after, or 
simultaneously with the display of that word. The word-by word 

15 facility is useful for partially sighted people as the visual 
display can handle words in large characters, up to the width of 
the screen. The word display or highlighting can be centred, such 
that the person can retain focus on one point on the screen while 
the words are displayed sequentially. This permits rapid reading, 

20 since the eyes' sacade movements are eliminated, and the time 
spent in backtracking and re-reading is avoided. The synchronism 
with speech reinforces the association between the written and 
spoken word, helpful for language learners and dyslexics. 

25 It should be understood that the output unit of the present 

invention may rest on a surface, or may be clipped to a part of 
a wearer's apparel, such as a belt, or may be strapped to and 
operated by one hand. 

30 Moreover the central processing unit may be remote from the 

input /output and may even be accessed by telephone and modem. 

In a further modification of the input/output units previously 
described, the units are provided with a transmitter, for example 
35 an infra-red or a radio transmitter, or a transmitter and 
receiver, providing a remote control link with an appliance or 
system such as a television, a kiosk or an automatic telling 
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machine, thus enabling the user to control the appliance or 
system using a dedicated pattern of pin or key operation and have 
from an audio or tactile output from the central processing unit 
confirmation of the control command. 
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CLAIMS 

5 

1. Sensory communication apparatus comprising a central 
processing unit having a data store, an output processor, means 
for controlling the output of data from the data store to the 
output processor, and output means connected to receive data from 

10 the processor, the output means being responsive to the output 
data whereby the output of the computer can be determined by one 
or more of the user's senses of touch, sound or sight. 

2. Apparatus as claimed in claim l, in which the means for 
15 controlling the output of the data store comprises a pointing 

device. 

3. Apparatus as claimed in claim 1 or claim 2, in which the 
output means includes tactile elements in the form of pins, each 

20 having associated with it an electro-mechanical transducer. 

4. Apparatus as claimed in claim 3, in which the pins are also 
associated with respective switches which are connected through 
an input processor to the central processing unit. 

25 

5. Apparatus as claimed in claim 3 or claim 4, in which the 
pins are arranged in pairs each for simultaneous contact by a 
respective finger and each pin of each pair can be vibrated at 
different frequencies or with different pulse lengths. 

30 

6. Apparatus as claimed in any of claims 3 to 5, in which the 
output means is operable in either a character mode or a surface 
mode. 

35 7. Apparatus as claimed in any of claims 3 to 6, in which the 
patterns in which the pins are actuated are related to text 
characters . 
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8. Apparatus as claimed in any preceding claim, including a 
speech synthesiser responsive to data output from the central 
processing unit. 

9. Apparatus as claimed in any of claims 1 to 3, or any of 
claims 5 to 8 when independent of claim 4, in which a keyboard 
or keypad is connected through an input processor to the central 
processing unit. 

10. Apparatus as claimed in clain 9, in which the keyboard or 
keypad is operable in a character mode or a control mode. 
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